{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Keras基础\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 一、什么是Keras\n",
    "\n",
    "<img src=\"keras.jpeg\" style=\"zoom:50%\" >\n",
    "\n",
    "Keras是一个高层神经网络API，Keras由纯Python编写而成并基Tensorflow、Theano以及CNTK后端。Keras 为支持快速实验而生，能够把你的idea迅速转换为结果，keras的基本特点：\n",
    "\n",
    "1. 简易和快速的原型设计（keras具有高度模块化，极简，和可扩充特性）\n",
    "2. 支持CNN和RNN，或二者的结合\n",
    "3. 无缝CPU和GPU切换\n",
    "\n",
    ">注意：Keras适用的Python版本是：Python 2.7-3.6\n",
    "\n",
    "在这里我们使用tensorflow作为keras的后端，实际上tensorflow已经将keras集成进了tensorflow中，所以keras可以看作是tensorflow的一个高阶API。\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tensorflow.python import keras\n",
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "### 二、Keras的重要API\n",
    "\n",
    "Keras的API主要可以分为以下几类：\n",
    "\n",
    "`layers`:主要涉及神经网络的一些构件，如全连接，卷积，最大池化等。\n",
    "\n",
    "`activations`:主要涉及神经网络的常用激活函数。\n",
    "\n",
    "`losses`:主要涉及一些常用的损失函数。\n",
    "\n",
    "`metrics`:主要涉及常用的模型评估方法。\n",
    "\n",
    "`optimizers`:主要涉及常用的优化方法。\n",
    "\n",
    "`models`:主要涉及模型的组装，模型的保存，模型的加载。\n",
    "\n",
    "`backend`: 主要涉及更底层的一些操作，如张量的操作。\n",
    "\n",
    "`callbacks`:主要涉及一些回调函数的操作。如模型的保存，模型的可视化。\n",
    "\n",
    "---\n",
    "\n",
    "使用Keras建立神经网络主要包含三个步骤：\n",
    "\n",
    "1. 准备数据。\n",
    "2. 搭建神经网络结构。\n",
    "3. 编译模型（确定损失函数、优化方法和回调函数）。\n",
    "4. 训练模型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tensor(\"input_11:0\", shape=(?, 28), dtype=float32)\n",
      "Tensor(\"input_12:0\", shape=(?, 56), dtype=float32)\n",
      "Tensor(\"concatenate_3/concat:0\", shape=(?, 84), dtype=float32)\n"
     ]
    }
   ],
   "source": [
    "## layers相关API\n",
    "input_tensor = keras.layers.Input(shape=(784,))\n",
    "\n",
    "keras.layers.Dense(units=10,activation=keras.activations.relu,use_bias=True)\n",
    "\n",
    "keras.layers.Softmax(axis=-1)(input_tensor)\n",
    "\n",
    "keras.layers.Flatten()\n",
    "\n",
    "a = keras.layers.Input(shape=(28,))\n",
    "print(a)\n",
    "b = keras.layers.Input(shape=(56,))\n",
    "print(b)\n",
    "c= keras.layers.concatenate([a,b],axis=-1)\n",
    "print(c)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function tensorflow.python.keras.activations.tanh(x)>"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## activations相关API\n",
    "keras.activations.relu\n",
    "keras.activations.sigmoid\n",
    "keras.activations.tanh"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function tensorflow.python.keras.losses.mean_squared_error(y_true, y_pred)>"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## losses相关API\n",
    "keras.losses.categorical_crossentropy\n",
    "keras.losses.sparse_categorical_crossentropy\n",
    "keras.losses.mean_squared_error\n",
    "keras.losses.mean_absolute_error\n",
    "keras.losses.logcosh()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.metrics.Recall at 0xb3e9ae470>"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## metric相关API\n",
    "keras.metrics.accuracy\n",
    "keras.metrics.sparse_top_k_categorical_accuracy\n",
    "keras.metrics.Precision()\n",
    "keras.metrics.Recall()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /Users/wanjun/anaconda/envs/python36/lib/python3.6/site-packages/tensorflow/python/ops/resource_variable_ops.py:435: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Colocations handled automatically by placer.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.optimizers.Adam at 0xb377ee278>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## optimizers相关API\n",
    "keras.optimizers.SGD()\n",
    "keras.optimizers.RMSprop()\n",
    "keras.optimizers.Adam()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /Users/wanjun/anaconda/envs/python36/lib/python3.6/site-packages/tensorflow/python/ops/resource_variable_ops.py:435: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Colocations handled automatically by placer.\n",
      "(None, 10)\n",
      "WARNING:tensorflow:From /Users/wanjun/anaconda/envs/python36/lib/python3.6/site-packages/tensorflow/python/keras/utils/losses_utils.py:170: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use tf.cast instead.\n"
     ]
    }
   ],
   "source": [
    "## models相关API\n",
    "# Sequential API\n",
    "model = keras.models.Sequential([keras.layers.Dense(30, activation=\"relu\",input_shape=[10]),keras.layers.Dense(1)])\n",
    "print(model.input_shape)\n",
    "model.compile(loss=\"mean_squared_error\", optimizer=\"sgd\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "<img src=\"structure1.png\" style=\"zoom:20%\" >"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.engine.training.Model at 0x156a48860>"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Function API\n",
    "#单输入单输出\n",
    "input_tensor = keras.layers.Input(shape=(12,))\n",
    "hidden1 = keras.layers.Dense(30, activation=\"relu\")(input_tensor)\n",
    "hidden2 = keras.layers.Dense(30, activation=\"relu\")(hidden1)\n",
    "concat = keras.layers.Concatenate(axis=-1)([input_tensor, hidden2])\n",
    "output_tensor = keras.layers.Dense(1)(concat)\n",
    "keras.models.Model(inputs=[input_tensor],outputs=[output_tensor])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "<img src=\"structure2.png\" style=\"zoom:20%\" >"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "#多输入单输出\n",
    "input_A = keras.layers.Input(shape=[5])\n",
    "input_B = keras.layers.Input(shape=[6])\n",
    "hidden1 = keras.layers.Dense(30, activation=\"relu\")(input_B)\n",
    "hidden2 = keras.layers.Dense(30, activation=\"relu\")(hidden1)\n",
    "concat = keras.layers.Concatenate(axis=-1)([input_A, hidden2])\n",
    "output = keras.layers.Dense(1)(concat)\n",
    "model = keras.models.Model(inputs=[input_A, input_B], outputs=[output])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "<img src=\"structure3.png\" style=\"zoom:40%\" >"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /Users/wanjun/anaconda/envs/python36/lib/python3.6/site-packages/tensorflow/python/keras/utils/losses_utils.py:170: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use tf.cast instead.\n"
     ]
    }
   ],
   "source": [
    "#多输入多输出\n",
    "input_A = keras.layers.Input(shape=[5])\n",
    "input_B = keras.layers.Input(shape=[6])\n",
    "hidden1 = keras.layers.Dense(30, activation=\"relu\")(input_B)\n",
    "hidden2 = keras.layers.Dense(30, activation=\"relu\")(hidden1)\n",
    "concat = keras.layers.concatenate([input_A, hidden2])\n",
    "output = keras.layers.Dense(1)(concat)\n",
    "aux_output = keras.layers.Dense(1)(hidden2)\n",
    "model = keras.models.Model(inputs=[input_A, input_B],outputs=[output, aux_output])\n",
    "model.compile(loss=[\"mse\", \"mae\"], loss_weights=[0.9, 0.1], optimizer=\"sgd\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***model的训练***\n",
    "\n",
    "```python\n",
    "model.fit([X_train_A, X_train_B], [y_train, y_train], epochs=20,batch_size=64,validation_data=([X_valid_A,X_valid_B],[y_valid, y_valid]))\n",
    "```\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function tensorflow.python.keras.engine.saving.load_model(filepath, custom_objects=None, compile=True)>"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## model系列API\n",
    "keras.models.save_model\n",
    "keras.models.load_model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function tensorflow.python.keras.backend.sum(x, axis=None, keepdims=False)>"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "## backend系列API\n",
    "keras.backend.argmax\n",
    "keras.backend.sum"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "## callback系列API\n",
    "keras.callbacks.ModelCheckpoint()\n",
    "keras.callbacks.TensorBoard()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 三、 纯Keras的神经网络\n",
    "\n",
    "下面将使用一个简单的纯keras来实现一个神经网络，纯keras实现神经网络方便且简单。\n",
    "\n",
    "要解决的问题fashion-mnist数据集问题:\n",
    "\n",
    "<img src=\"fashion-mnist.png\" style=\"zoom:50%\" >\n",
    "\n",
    "这是一个十分类问题。\n",
    "\n",
    "解决步骤:\n",
    "\n",
    "- 数据准备\n",
    "- 网络结构设计\n",
    "- 损失函数设计\n",
    "- 优化方法设计"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "import numpy as np\n",
    "from tensorflow.python import keras\n",
    "fashion_mnist = keras.datasets.fashion_mnist"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 数据准备\n",
    "(X_train_full, y_train_full), (X_test, y_test) = fashion_mnist.load_data()\n",
    "X_train_scaler = X_train_full/255.\n",
    "X_test_scaler = X_test/255.\n",
    "X_train_scaler = X_train_scaler.reshape(-1,28*28)\n",
    "X_test_scaler = X_test_scaler.reshape(-1,28*28)\n",
    "class_names = [\"T-shirt/top\", \"Trouser\", \"Pullover\", \"Dress\", \"Coat\",\"Sandal\", \"Shirt\", \"Sneaker\", \"Bag\", \"Ankle boot\"]\n",
    "y_train = keras.utils.to_categorical(y_train_full,num_classes=10)\n",
    "y_test = keras.utils.to_categorical(y_test,num_classes=10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0xb56e5b128>"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAADt9JREFUeJzt3V+MVOUZx/HfIwLyZ4OuKF0oSFFiVKLQIDegoRcS25BgL2rwiqZNtxc1aW9MjTeaNE20qbb1ppFaUpq0tjWWSkxTIU1be9GgizHIn2qRIGxZQUT55x+EfXqxZ5sV97zvMHNmzuDz/SRkd+eZd+bNWX47Z+Y557zm7gIQzyV1TwBAPQg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgLu3kk5kZhxMCbebu1sj9WnrlN7M7zew1M9trZve38lgAOsuaPbbfzCZIel3SHZIGJb0k6R53350Ywys/0GadeOVfJmmvu+9z9zOSfidpTQuPB6CDWgn/HEkHx/w8WNz2CWbWb2YDZjbQwnMBqFgrH/iNt2vxqd16d18vab3Ebj/QTVp55R+UNHfMz5+XdKi16QDolFbC/5KkhWb2BTObJGmtpM3VTAtAuzW92+/uZ83sXknPS5ogaYO776psZgDaqulWX1NPxnt+oO06cpAPgIsX4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0E1vUS3JJnZfkknJZ2TdNbdl1YxKQDt11L4C19y96MVPA6ADmK3Hwiq1fC7pC1mtt3M+quYEIDOaHW3f7m7HzKzqyVtNbN/u/sLY+9Q/FHgDwPQZczdq3kgs4cknXL3HyfuU82TASjl7tbI/Zre7TezaWbWM/q9pFWSdjb7eAA6q5Xd/lmSNpnZ6OP81t3/UsmsALRdZbv9DT0Zu/1A27V9tx/AxY3wA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QVBVX70XNLrmk/G/48PBwW5+7r68vWZ89e3Zpbfv27cmxEydOTNY//vjjZB1pvPIDQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFD0+T8DUr381DEAkrRq1apkvViXodSNN96YrK9YsaK0tnbt2uTYjz76KFmfPHlysp7aLrljBHLbLXf8xOOPP56s7927t7T2xBNPJMfmtkujeOUHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaCyS3Sb2QZJqyUdcfdFxW29kn4vab6k/ZLudvd3s0/GEt1tsWzZstLaDTfckBx77ty5ZP2aa65J1nfs2JGs33zzzaW13Pn8AwMDyfrRo0eT9TrlcvXII4+U1p5++unk2Nx2q3KJ7l9JuvO82+6X9Fd3Xyjpr8XPAC4i2fC7+wuSjp138xpJG4vvN0q6q+J5AWizZt/zz3L3IUkqvl5d3ZQAdELbj+03s35J/e1+HgAXptlX/sNm1idJxdcjZXd09/XuvtTdlzb5XADaoNnwb5a0rvh+naRnq5kOgE7Jht/MnpL0L0nXm9mgmX1T0sOS7jCz/0i6o/gZwEUk2+ev9Mku4j5/K9fGz50bft111yXruV59b29vae2qq65Kjj148GCyfvnll7dUP3ToUGlt5syZybE9PT3J+tatW5P106dPl9Z27dqVHJuzfPnyZD117IUkHT9+vLR25Ejpu2hJ0qZNm5L1Kvv8AD6DCD8QFOEHgiL8QFCEHwiK8ANB0eor5JaDTp36mmv1pS5fLUm33XZbsn7y5Mlk/b333iutzZo1Kzn2lltuSdZbXSY71a7bvXt3cuzg4GCyftlllyXrqdORU21ASTp8+HCyntuu06ZNS9ZT7d/cNr3vvvtKa8PDw7T6AKQRfiAowg8ERfiBoAg/EBThB4Ii/EBQHV+iu5VTY9sp11tN9bsXLFiQHHvrrbcm68eOnX991E/68MMPk/XZs2eX1qZOnZocm1oqWkofQyDlf2c33XRTaW3JkiXJsbfffnuynjsOIFWfMGFCcmxqm0rS22+/naznltFO/c5nzJiRHFsVXvmBoAg/EBThB4Ii/EBQhB8IivADQRF+IKiO9/lb6eW3cozA5MmTk/W+vr5k/corryyt5fr8uT597hLWuXPqU7363BLc77//frI+d+7cZD233V555ZXS2oEDB5Jjc5csz53Pn7p8du6y4Nu2bUvWc9stt91TvfzcMSepYxQu5PocvPIDQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFDZPr+ZbZC0WtIRd19U3PaQpG9JGj2p+QF3/3MDj6VLLy1/yjlz5iTHp5aDzl0nPdfXNUtf6jzVUz579mxybO6c+pzcueGpc89zS3S/9dZbyXpqiW0p3+dPLT+e64Xn1ivI/c5OnDhRWstdl3/SpEnJ+qJFi5L1d955J1k/depUaS13XEhqu1Xd5/+VpDvHuf0n7r64+JcNPoDukg2/u78gKX2pGQAXnVbe899rZjvMbIOZXVHZjAB0RLPh/7mkayUtljQk6dGyO5pZv5kNmNlAJ9cFBJDWVPjd/bC7n3P3YUm/kFR6BoW7r3f3pe6+NPcBDYDOaSr8Zjb2I96vStpZzXQAdEojrb6nJK2UNNPMBiU9KGmlmS2W5JL2S/p2G+cIoA2sk+/De3p6fPHixaX11atXJ8enerO5fnWub5vrOaf6/LnzynPnZ+fkjhNIHcOQu47BvHnzkvXc+Nx2Tx1/ketn79u3L1nPXbf/+PHjpbUpU6Ykx+auY5A7biR3DYbUOflvvvlmcuyTTz6ZrLt7Q++vOcIPCIrwA0ERfiAowg8ERfiBoAg/EFRHW31Tpkzx+fPnl9avv/765PhUPXeZ59ylllu5FHOulZdbgjvXZswtJ51qK+Xmlmun5Vqku3fvTtZT7bhcCzN3mnbqUu5Suh2Xa8/mTsltVeo08N7e3uTYF198MVmn1QcgifADQRF+ICjCDwRF+IGgCD8QFOEHgupon3/q1Km+cOHC0npqGWxJeuONN0prrSz9LeX74aleeq5Pn5tb7vTPXD871YvPjU1dDr2R8blTY1OXY8/12nPbNfc7O336dGntgw8+SI7N/U5Sl96WlLxEfa4+ffr05NgtW7Yk53X27Fn6/ADKEX4gKMIPBEX4gaAIPxAU4QeCIvxAUB3t85tZ8slmzpyZHL9gwYLS2owZM5Jjc+fE56QuYZ3rhef6/O0en5Lrd+f6+Lnz/VPLZOf+7+VWeMrNPXeNhpTcNm3luBAp/TvNHd+wc2f5GjlnzpzR8PAwfX4A5Qg/EBThB4Ii/EBQhB8IivADQRF+IKhsn9/M5kr6taTPSRqWtN7df2ZmvZJ+L2m+pP2S7nb3dzOP1bmDCoCgGr1ufyPh75PU5+4vm1mPpO2S7pL0dUnH3P1hM7tf0hXu/v3MYxF+oM0qW7TD3Yfc/eXi+5OS9kiaI2mNpI3F3TZq5A8CgIvEBb3nN7P5kpZI2iZplrsPSSN/ICRdXfXkALRP+kJjY5jZdEnPSPqeu5/IHXc9Zly/pP7mpgegXRo6scfMJkp6TtLz7v5Ycdtrkla6+1DxucDf3T250ibv+YH2q+w9v428xP9S0p7R4Bc2S1pXfL9O0rMXOkkA9Wnk0/4Vkv4p6VWNtPok6QGNvO//g6R5kg5I+pq7J9ei5pUfaL/KWn1VIvxA+1W22w/gs4nwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EFQ2/GY218z+ZmZ7zGyXmX23uP0hM/uvmb1S/PtK+6cLoCrm7uk7mPVJ6nP3l82sR9J2SXdJulvSKXf/ccNPZpZ+MgAtc3dr5H6XNvBAQ5KGiu9PmtkeSXNamx6Aul3Qe34zmy9piaRtxU33mtkOM9tgZleUjOk3swEzG2hppgAqld3t//8dzaZL+oekH7r7H81slqSjklzSDzTy1uAbmcdgtx9os0Z3+xsKv5lNlPScpOfd/bFx6vMlPefuizKPQ/iBNms0/I182m+Sfilpz9jgFx8EjvqqpJ0XOkkA9Wnk0/4Vkv4p6VVJw8XND0i6R9Jijez275f07eLDwdRj8coPtFmlu/1VIfxA+1W22w/gs4nwA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QVPYCnhU7KunNMT/PLG7rRt06t26dl8TcmlXl3K5p9I4dPZ//U09uNuDuS2ubQEK3zq1b5yUxt2bVNTd2+4GgCD8QVN3hX1/z86d069y6dV4Sc2tWLXOr9T0/gPrU/coPoCa1hN/M7jSz18xsr5ndX8ccypjZfjN7tVh5uNYlxopl0I6Y2c4xt/Wa2VYz+0/xddxl0mqaW1es3JxYWbrWbddtK153fLffzCZIel3SHZIGJb0k6R53393RiZQws/2Slrp77T1hM7td0ilJvx5dDcnMfiTpmLs/XPzhvMLdv98lc3tIF7hyc5vmVray9NdV47arcsXrKtTxyr9M0l533+fuZyT9TtKaGubR9dz9BUnHzrt5jaSNxfcbNfKfp+NK5tYV3H3I3V8uvj8paXRl6Vq3XWJetagj/HMkHRzz86C6a8lvl7TFzLabWX/dkxnHrNGVkYqvV9c8n/NlV27upPNWlu6abdfMitdVqyP8460m0k0th+Xu/kVJX5b0nWL3Fo35uaRrNbKM25CkR+ucTLGy9DOSvufuJ+qcy1jjzKuW7VZH+AclzR3z8+clHaphHuNy90PF1yOSNmnkbUo3OTy6SGrx9UjN8/k/dz/s7ufcfVjSL1TjtitWln5G0m/c/Y/FzbVvu/HmVdd2qyP8L0laaGZfMLNJktZK2lzDPD7FzKYVH8TIzKZJWqXuW314s6R1xffrJD1b41w+oVtWbi5bWVo1b7tuW/G6loN8ilbGTyVNkLTB3X/Y8UmMw8wWaOTVXho54/G3dc7NzJ6StFIjZ30dlvSgpD9J+oOkeZIOSPqau3f8g7eSua3UBa7c3Ka5la0svU01brsqV7yuZD4c4QfExBF+QFCEHwiK8ANBEX4gKMIPBEX4gaAIPxAU4QeC+h+83dx8O5GV/gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from matplotlib import pyplot as plt\n",
    "%matplotlib inline\n",
    "index = 666\n",
    "plt.imshow(X_train_scaler[666].reshape(28,28),cmap='gray')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "./keras_logs/run_2019_08_17-18_17_04\n",
      "Train on 60000 samples, validate on 10000 samples\n",
      "WARNING:tensorflow:From /Users/wanjun/anaconda/envs/python36/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use tf.cast instead.\n",
      "Epoch 1/5\n",
      "60000/60000 [==============================] - 9s 144us/sample - loss: 0.9584 - accuracy: 0.6643 - val_loss: 0.6506 - val_accuracy: 0.7607\n",
      "Epoch 2/5\n",
      "60000/60000 [==============================] - 9s 147us/sample - loss: 0.5625 - accuracy: 0.8038 - val_loss: 0.5463 - val_accuracy: 0.8077\n",
      "Epoch 3/5\n",
      "60000/60000 [==============================] - 9s 146us/sample - loss: 0.5009 - accuracy: 0.8243 - val_loss: 0.5042 - val_accuracy: 0.8240\n",
      "Epoch 4/5\n",
      "60000/60000 [==============================] - 9s 145us/sample - loss: 0.4666 - accuracy: 0.8352 - val_loss: 0.4811 - val_accuracy: 0.8324\n",
      "Epoch 5/5\n",
      "60000/60000 [==============================] - 9s 148us/sample - loss: 0.4437 - accuracy: 0.8425 - val_loss: 0.4705 - val_accuracy: 0.8300\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0xb57a762b0>"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from tensorflow.python.keras.layers import Dense,Softmax,Activation,Input\n",
    "from tensorflow.python.keras.models import Model\n",
    "from tensorflow.python.keras.callbacks import ModelCheckpoint,TensorBoard\n",
    "from tensorflow.python.keras import backend as K\n",
    "import os\n",
    "import tensorflow as tf\n",
    "\n",
    "# 网络架构设计\n",
    "input_tensor = Input(shape=(784,))\n",
    "hidden1 = Activation(activation='relu')(Dense(units=32,use_bias=True)(input_tensor))\n",
    "hidden2 = Activation(activation='relu')(Dense(units=16,use_bias=True)(hidden1))\n",
    "logits = Dense(units=10,use_bias=True)(hidden2)\n",
    "pred = Softmax()(logits)\n",
    "\n",
    "# 损失函数设计: 注意loss和metric的区别\n",
    "loss_op = keras.losses.categorical_crossentropy\n",
    "\n",
    "# 模型评估\n",
    "def accuracy(y_true,y_pred):\n",
    "    y_t = tf.argmax(y_true,axis=1)\n",
    "    y_p = tf.argmax(y_pred,axis=1)\n",
    "    return tf.reduce_mean(tf.cast(tf.equal(y_t,y_p),tf.float32))\n",
    "\n",
    "\n",
    "\n",
    "# 优化方法设计\n",
    "train_op = keras.optimizers.sgd()\n",
    "\n",
    "# 保存模型\n",
    "savemodel = ModelCheckpoint(filepath='keras_model/model_dnn.h5')\n",
    "\n",
    "# 模型组装和编译\n",
    "model = Model(inputs=[input_tensor],outputs=[pred])\n",
    "model.compile(loss=[loss_op],optimizer=train_op,metrics=[accuracy])\n",
    "\n",
    "\n",
    "# 可视化\n",
    "root_logdir = os.path.join(os.curdir, \"keras_logs\")\n",
    "def get_run_logdir(): \n",
    "    import time\n",
    "    run_id = time.strftime(\"run_%Y_%m_%d-%H_%M_%S\") \n",
    "    return os.path.join(root_logdir, run_id)\n",
    "run_logdir = get_run_logdir()\n",
    "print(run_logdir)\n",
    "tensorboard_cb = TensorBoard(run_logdir)\n",
    "\n",
    "# 模型的训练\n",
    "model.fit(X_train_scaler,y_train,epochs=5,validation_data=(X_test_scaler,y_test),callbacks=[savemodel,tensorboard_cb])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "input_3 (InputLayer)         (None, 784)               0         \n",
      "_________________________________________________________________\n",
      "dense_6 (Dense)              (None, 32)                25120     \n",
      "_________________________________________________________________\n",
      "activation_4 (Activation)    (None, 32)                0         \n",
      "_________________________________________________________________\n",
      "dense_7 (Dense)              (None, 16)                528       \n",
      "_________________________________________________________________\n",
      "activation_5 (Activation)    (None, 16)                0         \n",
      "_________________________________________________________________\n",
      "dense_8 (Dense)              (None, 10)                170       \n",
      "_________________________________________________________________\n",
      "softmax_2 (Softmax)          (None, 10)                0         \n",
      "=================================================================\n",
      "Total params: 25,818\n",
      "Trainable params: 25,818\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "# 查看参数数量和结构\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<tensorflow.python.keras.engine.input_layer.InputLayer at 0xb39e3b668>,\n",
       " <tensorflow.python.keras.layers.core.Dense at 0xb39e3b860>,\n",
       " <tensorflow.python.keras.layers.core.Activation at 0xb39e3b780>,\n",
       " <tensorflow.python.keras.layers.core.Dense at 0xb39e3ba58>,\n",
       " <tensorflow.python.keras.layers.core.Activation at 0xb39e3b7b8>,\n",
       " <tensorflow.python.keras.layers.core.Dense at 0xb39e6a470>,\n",
       " <tensorflow.python.keras.layers.advanced_activations.Softmax at 0xb39e3ba90>]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 获取layer的名字\n",
    "model.layers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[array([[ 0.04673344,  0.01502634, -0.07474566, ...,  0.06890095,\n",
       "         -0.04112365, -0.00896782],\n",
       "        [-0.00518269, -0.00286345,  0.08053879, ...,  0.07182655,\n",
       "         -0.01299394, -0.06554333],\n",
       "        [ 0.08002955, -0.03641549,  0.02303342, ...,  0.0393118 ,\n",
       "          0.05300613, -0.03454221],\n",
       "        ...,\n",
       "        [-0.0828051 ,  0.04777583, -0.02087372, ...,  0.07305664,\n",
       "          0.06184046,  0.04000294],\n",
       "        [ 0.01623803, -0.03462012, -0.0404018 , ...,  0.07255101,\n",
       "          0.01709375,  0.01892226],\n",
       "        [-0.07676131, -0.00633865,  0.07296253, ..., -0.00274478,\n",
       "          0.08141113, -0.00275608]], dtype=float32),\n",
       " array([-0.13452972,  0.02142563, -0.02201729,  0.1591276 , -0.11847489,\n",
       "         0.12031773, -0.01768341, -0.01737609,  0.00054321, -0.1729853 ,\n",
       "         0.04432423, -0.01464177,  0.32178038,  0.25212777,  0.19860578,\n",
       "         0.1715567 , -0.10305091, -0.16290314,  0.02499877,  0.12938379,\n",
       "         0.24135025, -0.05990688,  0.0913938 ,  0.20561   ,  0.07127212,\n",
       "         0.00596112, -0.04965558, -0.01646159,  0.07282738,  0.02870943,\n",
       "         0.11165093,  0.10920808], dtype=float32)]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 查看参数\n",
    "model.layers[1].get_weights()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dress\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.python.keras.models import load_model\n",
    "model = load_model('keras_model/model_dnn.h5')\n",
    "index = 666\n",
    "img = X_test_scaler[index].reshape(1,-1)\n",
    "pred = model.predict(img)\n",
    "print(class_names[np.argmax(pred)])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0xb54f98b00>"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAD01JREFUeJzt3WGMVfWZx/Hf4zAjCJg4onSc4sI2ZrNGs1QnxITNxk1jtU0TqKYGYiprmtIXrW6TvljFF/imidlsaX2xaUIXUoytbZNW4QVZMWYTt8mmcUQDuOxSlAFHRkYiUBCZYZhnX8yhmcW5///lnnvvuTPP95OYmbnPPXMfL/Obc+c+55y/ubsAxHNN1Q0AqAbhB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8Q1Lx2PpiZcThhA/r7+5P1sbGxmrWTJ082u52mWbhwYbLe29ubrL///vvNbGfOcHer536lwm9mD0h6TlKXpH9z92fLfD/M7IknnkjWDx06VLO2bdu2ZrfTNHfeeWey/sgjjyTrjz/+eDPbCafhl/1m1iXpXyV9RdLtktab2e3NagxAa5X5m3+VpMPu/p67j0v6laQ1zWkLQKuVCX+/pOl/dA0Xt/0/ZrbRzAbNbLDEYwFosjJ/88/0psJn3tBz962Stkq84Qd0kjJ7/mFJy6Z9/XlJx8u1A6BdyoT/DUm3mdkKM+uRtE7Srua0BaDVrMyVfMzsq5J+oqlR33Z3/2Hm/rzsn8GePXuS9XvuuSdZv3jxYs3a8PBwctv7778/Wf/www+T9ZwtW7bUrD300EPJbXt6epL1U6dOJeuPPvpozdrg4Nx9C6otc3533y1pd5nvAaAaHN4LBEX4gaAIPxAU4QeCIvxAUIQfCKrUnP+qHyzonP/uu+9O1nfvTk9L58+fn6yfPn26Zi13zvyNN96YrL/wwgvJ+urVq5P1RYsW1aydOXMmue3ixYuT9dxxAKnjJ9atW5fcdjard87Pnh8IivADQRF+ICjCDwRF+IGgCD8QFKO+WeCpp55K1h977LGatXnz0iduTk5OJuu33nprsj4yMpKsX7hwoWbt0qVLyW03b96crL/88svJeupU57mMUR+AJMIPBEX4gaAIPxAU4QeCIvxAUIQfCIo5/xyQmtUfOXIkuW3u3z93HIBZXSPlGeVO2d2wYUOy/sorrzT82HMZc34ASYQfCIrwA0ERfiAowg8ERfiBoAg/EFSpVXrNbEjSWUmXJE24+0Azmpprrrkm/Ts2N0u/5ZZbkvXUrD33vXO9LViwIFnPGRsbq1nLXVZ87dq1yXpuzp+6lsHExERy2whKhb/w9+5+sgnfB0Ab8bIfCKps+F3SHjN708w2NqMhAO1R9mX/anc/bmY3S3rVzP7H3V+ffofilwK/GIAOU2rP7+7Hi4+jkl6StGqG+2x19wHeDAQ6S8PhN7OFZrb48ueSvizpQLMaA9BaZV72L5X0UjFmmifpl+7+703pCkDLNRx+d39P0t80sZc5q8w575J0/PjxZP3o0aM1a7lz5lPX1Zfy5/t3dXUl66lZe3d3d3LbY8eOJes57bxWxWzEqA8IivADQRF+ICjCDwRF+IGgCD8QVDPO6kPFUstonzyZPuEyd8pvblyWG2Om6rklut99991kHeWw5weCIvxAUIQfCIrwA0ERfiAowg8ERfiBoJjzt0GrTy0dHx9v+LFzl+7OnbJbZonv3OnE77zzTrKOctjzA0ERfiAowg8ERfiBoAg/EBThB4Ii/EBQzPnngNQy2Dm5OX9O7nz+1Pfv7e1Nbpu7ZHkOl+5OY88PBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0Fl5/xmtl3S1ySNuvsdxW29kn4tabmkIUkPu/up1rU5u7V63nz99dfXrH3yySfJbcvM6euROp//9OnTyW37+/uT9VOn0j9yuTUJoqvnX/bnkh644rYnJb3m7rdJeq34GsAskg2/u78u6eMrbl4jaUfx+Q5Ja5vcF4AWa/Q13VJ3H5Gk4uPNzWsJQDu0/Nh+M9soaWOrHwfA1Wl0z3/CzPokqfg4WuuO7r7V3QfcfaDBxwLQAo2Gf5ekDcXnGyTtbE47ANolG34ze1HSf0n6KzMbNrNvSXpW0n1m9kdJ9xVfA5hFsn/zu/v6GqUvNbmXOSs3S88dB7B+fa1/gilnz56tWbt48WJy21afz1/msZ9++ulkPfe8II0j/ICgCD8QFOEHgiL8QFCEHwiK8ANBWTsvb2xmXEu5AXv37k3Wly9fXrN27ty55Lbz5rX2CO+JiYmatZ6enuS2S5cuTdbLjBnnMnev64lhzw8ERfiBoAg/EBThB4Ii/EBQhB8IivADQTHnnwVSp+xK0oULF2rWPv300+S23d3dDfV0We7nJ1XPXVr7uuuuS9bvuuuuZP3o0aPJ+lzFnB9AEuEHgiL8QFCEHwiK8ANBEX4gKMIPBNXy5bpQ3uhozQWRJEmLFi2qWSs7xy97znxq+9ycP3e+/0033ZSsR53z14s9PxAU4QeCIvxAUIQfCIrwA0ERfiAowg8ElZ3zm9l2SV+TNOrudxS3PSPp25I+Ku62yd13t6rJua6rqytZnz9/frKem5entHKOX/b755bwzj0vSKtnz/9zSQ/McPuP3X1l8R/BB2aZbPjd/XVJH7ehFwBtVOZv/u+Z2T4z225mNzStIwBt0Wj4fyrpC5JWShqR9KNadzSzjWY2aGaDDT4WgBZoKPzufsLdL7n7pKSfSVqVuO9Wdx9w94FGmwTQfA2F38z6pn35dUkHmtMOgHapZ9T3oqR7JS0xs2FJmyXda2YrJbmkIUnfaWGPAFogG353Xz/Dzdta0EtYqfPxpfxxAFXO+Vv5/XNz/tzzgjSO8AOCIvxAUIQfCIrwA0ERfiAowg8ExaW7O8ClS5eS9dwy2KlxWjuXYG/347d6TDnXsecHgiL8QFCEHwiK8ANBEX4gKMIPBEX4gaCY83eAc+fOJetjY2PJ+oIFC2rWcscQ5ORm6WWOQcidsjsxMZGsnz9/PllHGnt+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiKOf8s0NfXl6ynjhPIzflzs/Yyc/yc3KW3x8fHk/WhoaGGHxvs+YGwCD8QFOEHgiL8QFCEHwiK8ANBEX4gqOyc38yWSXpe0uckTUra6u7PmVmvpF9LWi5pSNLD7n6qda3GdeTIkWS9v7+/Zi13znt3d3dDPdUrtXx4bs5/5syZZH10dLShnjClnj3/hKQfuPtfS7pH0nfN7HZJT0p6zd1vk/Ra8TWAWSIbfncfcfe9xednJR2U1C9pjaQdxd12SFrbqiYBNN9V/c1vZsslfVHSHyQtdfcRaeoXhKSbm90cgNap+9h+M1sk6beSvu/uf6r3mG4z2yhpY2PtAWiVuvb8ZtatqeD/wt1/V9x8wsz6inqfpBnffXH3re4+4O4DzWgYQHNkw29Tu/htkg66+5ZppV2SNhSfb5C0s/ntAWiVel72r5b0TUn7zezt4rZNkp6V9Bsz+5akY5K+0ZoWsX///mR9xYoVNWvz5pU7a7vspbtT9Vxvvb29yTrKyf5kuPvvJdX6CfhSc9sB0C4c4QcERfiBoAg/EBThB4Ii/EBQhB8Iikt3t0Hu1NXc5bX37duXrD/44IM1a2UvvV12+1T92muvTW47ODiYrKMc9vxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBRz/jbIzcpzDhw4kKynjhPILcGdq5dd4ju1favn/KnjK3L/XxGw5weCIvxAUIQfCIrwA0ERfiAowg8ERfiBoJjzzwJvvfVWsn7x4sWatdwcPid3LYIyxzDktj106FDD37ue7x8de34gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCCo75zezZZKel/Q5SZOStrr7c2b2jKRvS/qouOsmd9/dqkZns9ysfXJyMlkfGhpK1s+fP1+zlruu/sTERLI+NjaWrOfOyU/N2nPf++DBg8k6yqnnIJ8JST9w971mtljSm2b2alH7sbv/S+vaA9Aq2fC7+4ikkeLzs2Z2UFJ/qxsD0FpX9Te/mS2X9EVJfyhu+p6Z7TOz7WZ2Q41tNprZoJmx9hLQQeoOv5ktkvRbSd939z9J+qmkL0haqalXBj+aaTt33+ruA+4+0IR+ATRJXeE3s25NBf8X7v47SXL3E+5+yd0nJf1M0qrWtQmg2bLht6m3i7dJOujuW6bd3jftbl+XlL7ELICOUs+7/aslfVPSfjN7u7htk6T1ZrZSkksakvSdlnQ4B7T61NLx8fGatb6+vpq13LZSfpSXkxrn9fT0JLf94IMPSj12asSaG69GUM+7/b+XNNOwmJk+MItxhB8QFOEHgiL8QFCEHwiK8ANBEX4gKC7d3Qatninv3LmzZi03S8+dNpu7dHfu/23ZsmU1a0uWLElue/jw4WQ9h0t3p7HnB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgrJ2zUDP7SNLRaTctkXSybQ1cnU7trVP7kuitUc3s7S/c/aZ67tjW8H/mwc0GO/Xafp3aW6f2JdFbo6rqjZf9QFCEHwiq6vBvrfjxUzq1t07tS6K3RlXSW6V/8wOoTtV7fgAVqST8ZvaAmf2vmR02syer6KEWMxsys/1m9nbVS4wVy6CNmtmBabf1mtmrZvbH4uOMy6RV1NszZvZB8dy9bWZfrai3ZWb2H2Z20MzeMbN/LG6v9LlL9FXJ89b2l/1m1iXpkKT7JA1LekPSenf/77Y2UoOZDUkacPfKZ8Jm9neSzkl63t3vKG77Z0kfu/uzxS/OG9z9nzqkt2cknat65eZiQZm+6StLS1or6R9U4XOX6OthVfC8VbHnXyXpsLu/5+7jkn4laU0FfXQ8d39d0sdX3LxG0o7i8x2a+uFpuxq9dQR3H3H3vcXnZyVdXlm60ucu0Vclqgh/v6T3p309rM5a8tsl7TGzN81sY9XNzGBpsWz65eXTb664nytlV25upytWlu6Y566RFa+brYrwz7T6TyeNHFa7+12SviLpu8XLW9SnrpWb22WGlaU7QqMrXjdbFeEfljT9wm6fl3S8gj5m5O7Hi4+jkl5S560+fOLyIqnFx9GK+/mzTlq5eaaVpdUBz10nrXhdRfjfkHSbma0wsx5J6yTtqqCPzzCzhcUbMTKzhZK+rM5bfXiXpA3F5xsk1b56Z5t1ysrNtVaWVsXPXaeteF3JQT7FKOMnkrokbXf3H7a9iRmY2V9qam8vTV3Z+JdV9mZmL0q6V1NnfZ2QtFnSy5J+I+lWScckfcPd2/7GW43e7tXUS9c/r9x8+W/sNvf2t5L+U9J+SZcvL7xJU39fV/bcJfparwqeN47wA4LiCD8gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0H9H2D3xMYT/pc/AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from matplotlib import pyplot as plt\n",
    "import numpy as np\n",
    "%matplotlib inline\n",
    "index = 666\n",
    "img = X_test_scaler[index]\n",
    "plt.imshow(img.reshape(28,28),cmap='gray')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "## 四、Keras中的现成模型\n",
    "\n",
    "`/site-packages/keras_applications/`\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function tensorflow.python.keras.applications.keras_modules_injection.<locals>.wrapper(*args, **kwargs)>"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf.keras.applications.inception_v3.InceptionV3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "input_shape=(224,224,1)\n",
    "base_model = tf.keras.applications.inception_v3.InceptionV3(include_top=True, weights=None, input_shape=input_shape, classes=100)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "x=base_model.output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "TensorShape([Dimension(None), Dimension(100)])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
